This invention relates to a system and method for operating a motor vehicle, such as an automobile, truck, aircraft or other vehicle, wherein a computer or computerized system is employed to assist and/or supplement the driver in the movement of the vehicle along a path of travel, such as a street or roadway and may be used to avoid obstacles and accidents.
A major cause of human suffering is automobile accidents. Approximately 49,000 people die in traffic accidents each year in the United States, and another three million are injured. The costs of death and injury accidents are staggering. According to the United States National Highway Traffic Safety Administration, crash damage and medical bills total $137 billion a year.
Automobile designers offer many safety features, including passenger restraints, improved braking systems, and body designs, intended to better protect automobile crash victims. But very little has been done in the area of automatic vehicle control systems based on modern electronics, computer systems, and advanced real-time software. This is true despite rapidly increasing capabilities in these technologies and pervasive application in many other areas including, for example the business, entertainment, and medical fields. Vehicle guidance and control technology has, of course, been applied with great success in military defense systems, avionics systems and space exploration systems. But, this technology is costly and has not been commercialized.
The opportunity exists today to develop cost effective, commercial automated vehicle control systems. New advances in low-cost hardware and software technology make implementation feasible. High-speed, parallel computer architectures, specialized image-processing equipment, and advanced special computers such as math coprocessors are available. Advanced expert system implementations based on concepts such as fuzzy logic and neural networks, and new, improved scanning systems for sensing environments around moving vehicles make it very timely, indeed, to pursue new approaches.
Work on these problems has begun. Intelligent vehicle/highway systems are being investigated with traffic control systems intended to minimize congestion. Vehicle location systems such as GPS (Global Positioning System) and route guidance systems are also being pursued. Certain systems for automated vehicle control have been proposed, including systems that scan the roadway directly ahead of a vehicle using radar/lidar or television and attempt to warn a driver of impending danger. Fuzzy logic expert systems for controlling vehicle speed (braking and throttle) based on scanning the roadway ahead of a vehicle have been described. Road tracking with electronic vehicle guidance is being pursued. Fuzzy logic has been applied to braking systems in subway and train systems.
While these developments are important, they fail to protect vehicles from many types of collisions or minimize the damage therefrom. More particularly, such systems fail to exercise simultaneous, coordinated control over vehicle steering and speed, fail to take full advantage of identification of different obstacle or hazard types using standard stored models of production vehicles and other commonly encountered roadway objects, fail to deal effectively with objects and hazards located simultaneously on different sides of the vehicle, and fail to capitalize fully on modern expert system decision and control technology, such as represented by fuzzy logic and neural network methods, to deal with more complex hazardous situations.
In a preferred form of the invention, a video scanning system, such as a television camera and/or one or more laser scanners mounted on the vehicle scan the road in front of the vehicle and generate image information which is computer analyzed per se or in combination with a range sensing system to warn the driver of hazardous conditions during driving by operating a display, such as a heads-up display, and/or a synthetic speech generating means which generates sounds or words of speech to verbally indicate such road conditions ahead of the vehicle.
The preferred form of the invention provides audible and/or visual display means to cooperate in indicating to the driver of a motor vehicle both normal and hazardous road conditions ahead as well as driving variables such as distances to stationary objects, and other vehicles; the identification, direction of travel and speed of such other vehicles, and the identification of and distances to stationary or slowly moving objects such as barriers, center islands, pedestrians, parked cars poles, sharp turns in the road and other conditions. In addition, the image analyzing computer of the vehicle may be operated to scan and decode coded and/or character containing signs or signals generated by indicia or code generating other devices within or at the side of the road and indicating select road and driving conditions ahead.
The computer is operable to analyze video and/or other forms of image information generated as the vehicle travels to identify obstacles ahead of the vehicle and, in certain instances, quantify the distance between the vehicle containing same on the basis of the size of the identified vehicle or object and/or by processing received pulse-echo signals. Using such identifying information and comparing it with information on the shapes and sizes of various objects such as rear and front profiles of all production vehicles and the like and their relative sizes or select dimensions thereof, indications of distances to such objects may be computed and indicated as further codes. When the closing distance becomes hazardous, select vehicle subsystems may be automatically controlled by the computer as it continues to analyze image signals generated by the television camera. A first subsystem generates a first select code or codes which controls an electronic display, such as a heads-up display to cause it to display a warning indication, such as one or more flashing red light portions of the display or other lighted effect. For example, the display may project on the windshield or dashboard such information as images of the controlled vehicle and other vehicles in and adjacent its path of travel and relative distances thereto as well as groups of characters defining same, colored and flashing warning lights and the like for pre-warning and warning purposes. A second subsystem generates a code or series of codes which control a sound generating means which generates a select sound such as a horn, buzzing sound and/or select synthetic speech warning of the hazardous condition detected and, in certain instances, generating sounds of select words of speech which may warn of same and/or suggest corrective action (s) by the vehicle operator or driver to avoid an accident.
A third subsystem comes on-line and generates one or more codes which are applied to at least partly effect a corrective action such as by pulsing one or more motors or solenoids to apply the brakes of the vehicle to cause it to slow down. If necessary to avoid or lessen the effects of an accident, the third subsystem stops the forward travel of the vehicle in a controlled manner depending on the relative speeds of the two vehicles, and/or the controlled vehicle and a stationery object or structure and the distance therebetween.
A fourth subsystem, which may be part of or separate from the third subsystem may generate one or more codes which are applied to either effect partial and/or complete control of the steering mechanism for the vehicle to avoid an obstacle and/or lessen the effect of an accident. Either or both the third or fourth subsystem may also be operable to control one or more safety devices by controlling motors, solenoids or valves, to operate a restraining device or devices for the driver and passenger(s) of the vehicle, such as a safety belt tightening means, an air bag inflation means or other device designed to protect human beings in the vehicle.
The second, and/or third and fourth subsystems may also be operable to effect or control the operation of additional warning means such as the horn, headlights and/or other warning lights on the vehicle or other warning means which operates to alert, flag or warn the driver of the approaching or approached vehicle or a pedestrian of the approaching hazardous condition. One or more of these subsystems may also be operable to generate and transmit one or more codes to be received and used by the approaching or approached vehicle or a roadside device to effect additional on-line warning(s) of the hazardous condition, and/or may be recorded on a disc or RAM (random access memory) for future analysis, if necessary.
In a modified form of the invention, the vehicle warning system may also include a short wave receiving means to receive code signals from other vehicles and/or short wave transmitters at the side of or within the road for controlling the visual, audio and/or brake and steering means of the vehicle to avoid or lessen the effects of an accident and/or to maintain the vehicle in-lane and in proper operating condition as it travels.
The systems and methods of this invention preferably employ computerized image analyzing techniques of the types disclosed and defined in such patents of mine as U.S. Pat. Nos. 4,969,038 and 4,979,029 and references cited in the file wrappers thereof as well as other more recent patents and include the use of known artificial intelligence, neural networking and fuzzy logic computing electronic circuits.
While the invention is described herein principally in connection with an automobile on a roadway, it may be used in connection with controlling any powered vehicle, including a motor vehicle, a boat, a train, or an aircraft.
Accordingly it is a primary object of this invention to provide a new and improved system and method for controlling the operation of a powered vehicle.
Another object is provide a system and method for assisting the driver of a powered vehicle in controlling its operation to avoid an accident or hazardous driving condition.
Another object is to provide a system and method employing computerized image analysis to control or assist the driver of a motor vehicle in controlling its operation to avoid hazardous conditions such as collisions with other vehicles, stationery objects or pedestrians.
Another object is to provide a computerized system and method for controlling the speed of travel of a motor vehicle to lessen the chances of an accident while being driven by a person.
Another object is to provide a system and method employing a television scanning camera mounted on a vehicle for scanning the field ahead, such as the image of the road ahead of the vehicle and a computer for analyzing the image signals generated wherein automatic image intensifying, or infra-red scanning and detection means is utilized to permit scanning operations to be effected during driving at night and in low light, snowing or fog conditions.
Another object is to provide a system and method employing a television camera or other video scanning means mounted on a moving motor vehicle for scanning, detecting and identifying obstacles such as other vehicles ahead of such moving vehicle wherein the video image signals are analyzed to determine distances to such objects.
Another object is to provide a computer controlled safety system for a motor vehicle which employs a television camera and an auxiliary scanning means to both identify obstacles in the path of the vehicle and determine distance therefrom on a real time and continuous basis for use in warning the operator of same and/or in controlling the operation of the vehicle to avoid a collision.